User interface interaction system and method for handheld device and tv set

ABSTRACT

An interaction system and method for interaction between a handheld device and a television comprises the configuration and connection of a first HDMI transferring module and a television screen in addition to a configuration and connection of a second HDMI transferring module, a touch screen, a gesture managing module and a display managing module. A user interface system and method for interaction between a handheld device and a television can utilize a handheld device having a touch screen to operate the television without requiring the assistance of other hardware units, such as a mouse or keyboard. All the operations can be done by using only the handheld device, thereby reducing the costs of devices.

TECHNICAL FIELD

The present invention relates to a TV set system and a communicationmethod thereof, and in particular, relates to a UI interaction systemand method for a handheld device and a TV set.

BACKGROUND

Currently following the television developing trend of “highdefinition”, “networking” and “intelligence”, the smart televisionsappear. A real smart television, however, should have the capability ofobtaining TV programs through multiple channels such as network, an AVdevice and/or PC so as to display a program expected most by a customeron a large screen clearly by easy operation through simple andintegrated user interface. The future smart television will be anentertainment center through which a user can conduct a lot ofactivities such as searching TV channels, recording TV programs, playingcable television programs, conducting network video, watching movies,and/or playing games.

In a time of smart phones and smart televisions, people have morechoices and ways for entertainments, and many people hope that they canperform operations such as playing games or choosing and watchingfavorite moves directly on the television by a handheld device. UIinteraction design, that is, user interface interaction is a designpopular in recent years used for the interaction between people andtelevisions or computers, which can fully reflect the orientation andcharacters of a product by not only making the product unique andtasteful, but also making a user's operation comfortable, simple, andfree. As general televisions do not have a touch function, so currentlythe commonly used method for interacting with a television is using amouse or a keyboard. The above mentioned methods, however, can not meetthe requirements of a user on easy operation to a television through ahandhold device well, for instance, can not support the user in playinga game requiring sliding operations in a large area when an accurateoperation such as choosing a move or a song is conducted, thus bringingmany inconveniences to the user's television operation with a handhelddevice. Therefore, there is much room for the prior art to be improvedand developed.

SUMMARY OF THE INVENTION

Considering the above drawbacks in the prior art, an object of thepresent invention is to provide a UI interaction system and method for ahandheld device and a TV set to solve the inconvenience problem inoperating a television through a handheld device in the prior art.

A technical solution of the present invention is as below:

A user interface (UI) interaction method for a handheld device and a TVset, which comprises the steps of:

A. transferring EDID data of a television to a second HDMI transferringmodule through a first HDMI transferring module by the television;

B. transferring the EDID data of the television to a display managingmodule through the second HDMI transferring module by a handheld device;

C. initializing UI display data according to the EDID data andoutputting the UI display data to the television by the display managingmodule;

D. initializing an operation mode to a cursor mode by a gesture managingmodule;

E. receiving a touch event from a user, updating the UI display data andsending it to the television by the handheld device;

the operation mode comprises a cursor mode and a touch mode;

when the touch event is the press event, the step E comprises:

E11. determining whether to switch the operation mode according towhether coordinates of the press event are in an operation mode switcharea, and proceeding to the next step;

E12. judging whether the operation mode is the cursor mode, if yes,saving coordinates of the press event without sending the touch event,and if not, sending the touch event, updating the UI display data andsending it to the television;

when the touch event is the move event, the step E comprises:

E21. judging whether the operation mode is the cursor mode, if not,sending the touch event, updating the UI display data and sending it tothe television, and if yes, proceeding to the next step;

E22. judging whether a moving distance of the move event is bigger thanthe threshold, if not, not processing the move event, and if yes, thenentering a cursor moving state, calculating and moving a displayposition of the cursor, updating the UI display data and sending it tothe television.

In the user interface interaction method for a handheld device and a TVset, in the step E, when the touch event is the press event, and thecoordinates of the press event are in the operation mode switch area,the operation mode is then switched to the touch mode.

In the user interface interaction method for a handheld device and a TVset, the operation mode switch area is arranged at the top left of thetouch screen.

A user interface (UI) interaction system for a handheld device and a TVset comprises:

a television comprising a display screen and a first HDMI transferringmodule connected with the display screen;

a handheld device comprising a second HDMI transferring module, adisplay managing module, a gesture managing module and a touch screenconnected in sequence;

the second HDMI transferring module and the first HDMI transferringmodule are in a communicating connection, used for transferring EDIDdata of the television from the first HDMI transferring module to thedisplay managing module, and the touch screen is used for receiving atouch event performed by a user;

the display managing module is used for initializing UI display dataaccording to the EDID data, and transferring the UI display data to thetelevision, and the gesture managing module is used to manage anoperation mode.

In the user interface interaction system for a handheld device and a TVset, the operation mode comprises a cursor mode and a touch mode.

In the user interface interaction system for a handheld device and a TVset, the touch event comprises a press event and a move event.

In the user interface interaction system for a handheld device and a TVset, the touch screen is arranged with an operation mode switch area forswitching the operation mode.

In the user interface interaction system for a handheld device and a TVset, the operation mode switch area is arranged at the top left of thetouch screen.

A user interface (UI) interaction method for a handheld device and a TVset, which comprises the steps of:

A. transferring EDID data of a television to a second HDMI transferringmodule through a first HDMI transferring module by the television;

B. transferring the EDID data of the television to a display managingmodule through the second HDMI transferring module by a handheld device;

C. initializing UI display data according to the EDID data, andoutputting the UI display data to the television by the display managingmodule;

D. initializing an operation mode into a cursor mode by a gesturemanaging module; and

E. receiving a touch event from a user, updating the UI display data andsending it to the television by the handheld device.

In the user interface (UI) interaction method for a handheld device anda TV set, the operation mode comprises a cursor mode and a touch mode.

In the user interface (UI) interaction method for a handheld device anda TV set, the touch event in the step E comprises a press event and amove event.

In the user interface interaction method for a handheld device and a TVset, wherein when the touch event is the press event, the step Ecomprises:

E11. determining whether to switch the operation mode according towhether the coordinates of the press event are in an operation modeswitch area, and proceeding to the next step;

E12. judging whether the operation mode is the cursor mode, if yes,saving the coordinates of the press event without sending the touchevent, and if not, sending the touch event, updating the UI display dataand sending it to the television.

In the user interface interaction method for a handheld device and a TVset, when the touch event is the move event, the step E comprises:

E21. judging whether the operation mode is the cursor mode, if not,sending the touch event, updating the UI display data and sending it tothe television, and if yes, proceeding to the next step;

E22. judging whether a moving distance of the move event is larger thana threshold, if not, not processing the move event, and if yes, thenentering a cursor moving state, calculating and moving a displayposition of the cursor, updating the UI display data and sending it tothe television.

In the user interface interaction method for a handheld device and a TVset, in the step E, when the touch event is the press event, and thecoordinates of the press event are in the operation mode switch area,the operation mode is then switched into the touch mode.

In the user interface interaction method for a handheld device and a TVset, the operation mode switch area is arranged at the top left of thetouch screen.

Beneficial effects: a UI interaction system and method for a handhelddevice and a TV set of the present invention has the advantage ofexceedingly simple operation that can meet the requirements on anaccurate operation such as choosing a movie or a song only by operatingthe touch screen of the handheld device, as well as meet therequirements of a user on sliding on a large area when playing a game.This method allows full play to the display advantage of a televisionwith high resolution, and also takes the advantage of simple operationof a handheld device, thereby providing excellent mobile internetexperiences to the users. In addition to that, the present inventionuses a handheld device with a touch screen to operate a televisionwithout requiring the assistance of other hardware units, such as amouse or a keyboard, and all operations can be done with the handhelddevice only, thus reducing the costs of devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing the structure of a UI system forinteraction between a handheld device and a TV set of the presentinvention;

FIG. 2 is a flow chart of a UI method for interaction between a handhelddevice and a TV set of the present invention;

FIG. 3 is a flow chart showing the processing of a press event of a UImethod for interaction between a handheld device and a TV set accordingto the present invention;

FIG. 4 is a flow chart showing the processing of a move event of a UImethod for interaction between a handheld device and a TV set accordingto the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides a UI interaction system and method for ahandheld device and a TV set, which will be described in details asbelow in order to make the purposes, technical solutions and effects ofthe present invention more explicit and clearer. It should be understoodthat the exemplary embodiments described here are used for explainingthe present invention only with no intention to limit the scope of thepresent invention.

Referring to FIG. 1, which is a schematic diagram showing the structureof a UI interaction system for a handheld device and a TV set of thepresent invention. UI interaction design, that is, user interfaceinteraction, is a design popular in recent years used for theinteraction between people and machines, which can represent theorientation and characters of software fully by not only making aproduct unique and tasteful, but also making a user's operationcomfortable, simple, and free.

A UI interaction system for a handheld device and a TV set of thepresent invention is achieved through the communicating connectionbetween HDMI transferring modules arranged for a television 110 and ahandheld device 120. HDMI (High Definition Multimedia Interface) refersto a high definition multimedia interface, which is a digital video oraudio interface technology capable of transferring audio signals andvideo signals simultaneously. As shown in FIG. 1, the television 110mainly comprises a first HDMI transferring module and a display screen,while the handheld device 120 comprises a touch screen, a gesturemanaging module, a display managing module and a second HDMItransferring module connected in sequence.

To be specific, in a preferred embodiment of the present invention, thetelevision 110 is arranged with the first HDMI transferring module, thehandheld device 120 is equipped with the second HDMI transferringmodule, and a hardware interface used for HDMI transmission is set onthe handheld device 120, thus the second HDMI transferring module of thehandheld device 120 can be coupled to the first HDMI transferring moduleof the television 110 through a HDMI data cable in a communicatingconnection manner by a user. Under HDMI Protocol, the handheld device120 can obtain EDID data (Extended Display Identification Data) in thetelevision 110. EDID is a data format defined by a standard published byVESA (Video Electronics Standards Association), which comprisesparameters relating to performance of a TV including the informationsuch as the resolution of a television display screen, suppliers,maximum image size, color setting, and/or presetting of manufacturer.

The handheld device 120 can adjust its touch screen resolution so as tomake it matches with the display screen resolution of the television 110according to the obtained EDID data. In a preferred embodiment of thepresent invention, the handheld device 120 obtains the EDID data of thetelevision 110 in order to adjust the touch screen resolution of thehandheld device 120 first so as to improve experience effect of handhelddevice 120. A display managing module of the handheld device 120initializes UI display data (user interface display data) according tothe obtained EDID data such as the display screen resolution of thetelevision 110, and transfers the UI display data to the television 110through the second HDMI transferring module and the first HDMItransferring module, thereafter the television 110 displays the updatedframes synchronously according to the UI display data rather than justdisplaying the touch screen resolution of the handheld device 120 on thetelevision 110. The process mentioned above can ensure that the UIdisplay data transferred from the handheld device 120 to the television110 is the standard HDMI output thus to give full play to the displayadvantage of high resolution of the television 110, thereby providingthe best mobile internet experiences to users. An important character ofthe present invention is that the handheld device 120 can adapt totelevisions 110 with various parameters through the HDMI connectionbetween the handheld device 120 and the television 110. For instance,the users' televisions may be completely different in sizes and displayscreen resolutions; however, they can be assured of outputting highdefinition TVs by the UI display data transferred to them via thedisplay managing module and the second HDMI transferring module of thehandheld device 120.

As shown in FIG. 1, the handheld device 120 is further provided with agesture managing module and a touch screen. The gesture managing modulecan provides a switching function between a cursor mode and a touchmode, and accordingly an operation mode switch area is arranged on thetouch screen of the handheld device 120 for the user to switch anoperation mode. At the moment, in the aspect of hardware, it also needsto coordinate the point-point relation between the touch screen of thehandheld device 120 and the display screen of the television 110 byperforming a touch at a location of the touch screen and showing thetouch on the display screen at the corresponding location.

Referring to FIG. 2, which is a flow chart of a UI interaction methodfor a handheld device and a TV set of the present invention, as shown inthe Figure, the UI interaction method of the present invention mainlycomprises the steps of:

S01. transferring EDID data of a television 110 to a second HDMItransferring module through a first HDMI transferring module by thetelevision 110, during the course of which, the EDID data of thetelevision is first read by the CPU of the television 110, then the EDIDdata of the television 110 is read by the first HDMI transferringmodule, and thereafter transferred to the second HDMI transferringmodule for processing;

S02. transferring the EDID data of the television 110 to a displaymanaging module through the second HDMI transferring module by ahandheld device 120, the display managing module initializes UI displaydata after receiving the EDID data of the television 110;

S03. initializing UI display data, and outputting the UI display data tothe television 110 by the display managing module, that is the displaymanaging module can initialize the UI display data according to theinformation included in the EDID data such as the screen resolution ofthe television 110, then transfer the UI display data to the first HDMItransferring module through the second HDMI transferring module, and atlast have it displayed on the television 110 synchronously;

S04. initializing an operation mode by a gesture managing module, thedefault operation mode can be set as a cursor mode, under which a usercan locate the touch point of each minimum unit on the televisionaccurately, to be specific, a small icon will be shown on the televisionscreen to prompt the user for the current cursor location;

S05. receiving a touch event from the user by the touch screen, updatingthe UI display data and sending the updated UI display data to thetelevision 110 for responding, while sliding on the touch screen of thehandheld device 120 by the user will move the position of the cursor butwill not send the touch event to the current upper application such asvideo selection application for responding, after the user moves thecursor to a desired position, he or she can tap any area on the touchscreen, and now the touch event will be send to the upper application.During the course, there will be a small icon shown on the televisionscreen, the process of operating the television is similar to that ofoperating a computer, and the coordinates of the touch event sent by theuser are the coordinates of the cursor displayed on the televisionscreen rather than the coordinates of the position on the touch screentapped by the user. In addition, it is worth mentioning that in thepresent invention, the application program used by the user to performcontrol through the handheld device 120 is a built-in applicationprogram of the handheld device 120 rather than an application programrelevant or irrelevant to the television. The purpose of the presentinvention is to control the television 110 through the operationsperformed on the handheld device 120 and conduct displaying on thetelevision 110 so as to give full play to the display advantage of highresolution of the television 110 as well as take the advantage of simpleoperation of the handheld device 120.

When the user needs to perform other operations, for instance needs touse the continuous-touch function or the sensor function, he or she cantouch an operation mode switch area provided by the gesture managingmodule. The operation mode switch area can be arranged at the top leftof the touch screen, and the operation mode can be switched to the touchmode only by one tap conducted within the operation mode switch area andswitched back to the cursor mode when the user performs one tap again inthe operation mode switch area.

Specifically, as shown in FIG. 2, the gesture managing moduleinitializes the operation mode as a cursor mode in the step S04; andafter the step S05, the method further comprises:

S06. switching the operation mode to the touch mode, touching the touchscreen and sending the touch event performed by the user. When the userneeds to switch the operation mode to a touch mode, for instance whenthe user needs to perform sliding operations on a large area, he or shecan switch the operation mode to the touch mode by tapping the operationmode switch area preset on the touch screen, that is the touch event inthe step S05 is a press event with the coordinates in the operation modeswitch area, and now the operation mode is switched to the touch mode.Any touch operation such as tapping or sliding on the touch screen willbe sent to the upper application such as a game application forresponding. During the process, the television can respond to themovements according to the touch gesture, such as a single-point slidingor multi-point scaling, and the coordinates of the touch event sent bythe user are the coordinates of a position tapped or slide on the touchscreen by the user.

Continuing further, after a user sends a touch event by touching thetouch screen, the handheld device 120 will judge the type of the touchevent which includes a move event and a press event, and respondaccording to the type of the touch event correspondingly.

Referring to FIG. 3, which is a flow chart showing the processing of apress event of a UI interaction method for a handheld device and a TVset of the present invention, when the type of the touch event is apress event, as shown in the Figure, the step S05 comprises:

L01. determining whether the coordinates are in the operation modeswitch area, if yes, switching the operation mode and proceeding to thenext step, and if not, proceeding to the next step directly, in otherwords, after a press operation on the touch screen of the handhelddevice 120 is performed by the user, the handheld device 120 judgeswhether the coordinates of the press operation are in the operation modeswitch area, if the coordinates are in the operation mode switch area,then switching the operation mode, and if not, proceeding to the nextstep;

L02. judging whether the operation mode is a cursor mode, if yes, savingthe coordinates of the press event without sending the touch event, andif not, sending the touch event, updating the UI display data andsending it to the television 110. In the process, no matter what thecurrent operation mode of the handheld device 120 is, the handhelddevice 120 will first judge whether it is a cursor mode, if yes, savingthe coordinates of the press event without sending the touch event, andif not, sending the touch event, updating the UI display data andsending it to the television 110.

Referring to FIG. 4, which is a flow chart showing the processing of amove event of a UI interaction method for a handheld device and a TV setof the present invention, and when the touch event is a move event, asshown in the Figure, the step S05 comprises:

T01. judging whether it is a cursor mode, if not, sending the touchevent, updating and sending the UI display data to the television 110,and if yes, proceeding to the next step, that is when the user is movingon the touch screen, the handheld device judges first whether thecurrent operation mode is a cursor mode, if it is not a cursor mode,that is the current operation mode is a touch mode, then sending thetouch event, updating and sending the UI display data to the television,and if it is a cursor mode, then proceeding to the next step;

T02. judging whether a moving distance is bigger than the threshold, ifnot, not processing the move event, and if yes, then entering a cursormoving state, calculating and moving the display position of the cursor,updating and sending the UI display data to the television 110, that isin the cursor mode, the handheld device judges whether the movingdistance is bigger than the threshold first after the user performs amove operation, if not, then no processing is conducted, and if themoving distance is bigger than the threshold, then entering into acursor moving state, calculating and moving the display position of thecursor, thereafter, updating the UI display data and sending it to thetelevision.

A UI interaction method and system for a handheld device and a TV set ofthe present invention uses a handheld device with a touch screen tooperate a television without requiring assistance of other hardwareunits, such as a mouse or a keyboard, all operations can be done withthe handheld device only, thus to reduce the costs of the devices. Inaddition to that, the present invention has the advantage of exceedinglysimple operation, it can achieve data transmission and operation betweena handheld device and a TV set by using the existed HDMI interfaces, andit can meet the requirements on the accurate operation of such aschoosing a movie or a song only by operating the touch screen of thehandheld device, as well as meet the requirements of a user on slidingin a large area when playing a game. This method can allow full play tothe display advantage of a television with high resolution, and also cantake the advantage of simple operation of a handheld device, thusproviding excellent mobile internet experiences to users.

It should be understood that the application of the present invention isnot limited to above described embodiments, those who skilled in thefield may make improvements or variations according to the abovedescription, and all the improvements and variations should fall intothe protection scope of claims attached to the present invention.

1. A user interface (UI) interaction method for a handheld device and atelevision, which comprises the steps of: transferring EDID data(Extended Display Identification Data) of a television to a second HDMI(High Definition Multimedia Interface) transferring module through afirst HDMI transferring module by the television; transferring the EDIDdata of the television to a display managing module through the secondHDMI transferring module by a handheld device; initializing UI displaydata according to the EDID data and outputting the UI display data tothe television by the display managing module; initializing an operationmode to a cursor mode by a gesture managing module; receiving a touchevent from a user, updating the UI display data and sending it to thetelevision by the handheld device, wherein the operation mode comprisesa cursor mode and a touch mode, wherein when the touch event isreceiving the touch event from the user, the step of receiving a touchevent from the user comprises: determining whether to switch theoperation mode according to whether coordinates of the press event arein an operation mode switch area; deciding whether the operation mode isthe cursor mode, and if so, saving coordinates of the press eventwithout sending the touch event, and if not, sending the touch event,updating the UI display data and sending the UI display data to thetelevision, wherein when the touch event is a move event, the step ofreceiving a touch event from a user comprises: determining whether theoperation mode is the cursor mode, and if not, sending the touch event,updating the UI display data and sending the UI display data thetelevision; determining whether a moving distance of the move event islarger than a threshold, and if not, not processing the move event, andif so, then entering a cursor moving state, calculating and moving adisplay position of the cursor, updating the UI display data and sendingit to the television.
 2. The user interface interaction method for ahandheld device and a television according to claim 1, wherein in thestep of receiving a touch event from a user, when the touch event is thepress event, and the coordinates of the press event are in the operationmode switch area, the operation mode is switched to the touch mode. 3.The user interface interaction method for a handheld device and atelevision of claim 1, wherein the operation mode switch area isarranged at a top left of a touch screen of the handheld device.
 4. Auser interface (UI) interaction system for a handheld device and atelevision comprising: the television comprising a display screen and afirst HDMI (High Definition Multimedia Interface) transferring moduleconnected with the display screen; a handheld device comprising a secondHDMI transferring module, a display managing module, a gesture managingmodule and a touch screen connected in sequences, wherein the secondHDMI transferring module and the first HDMI transferring module are in acommunicating connection, used for transferring EDID data (ExtendedDisplay Identification Data) of the television from the first HDMItransferring module to the display managing module, and the touch screenis used for receiving a touch event performed by a user, and wherein thedisplay managing module is used for initializing UI display dataaccording to the EDID data, and transferring the UI display data to thetelevision, and the gesture managing module is used to manage anoperation mode.
 5. The user interface interaction system for a handhelddevice and a television of claim 4, wherein the operation mode comprisesa cursor mode and a touch mode.
 6. The user interface interaction systemfor a handheld device and a television of claim 4, wherein the touchevent comprises a press event and a move event.
 7. The user interfaceinteraction system for a handheld device and a television of claim 4,wherein the touch screen is arranged with an operation mode switch areafor switching the operation mode.
 8. The user interface interactionsystem for a handheld device and a television of claim 7, wherein theoperation mode switch area is arranged at a top left of the touchscreen.
 9. A user interface (UI) interaction method for a handhelddevice and a television, which comprises the steps of: transferring EDIDdata (Extended Display Identification Data) of a television to a secondHDMI (High Definition Multimedia Interface) transferring module througha first HDMI transferring module by the television; transferring theEDID data of the television to a display managing module through thesecond HDMI transferring module by a handheld device; initializing UIdisplay data according to the EDID data, and outputting the UI displaydata to the television by the display managing module; initializing anoperation mode into a cursor mode by a gesture managing module; andreceiving a touch event from a user, updating the UI display data andsending it to the television by the handheld device.
 10. The userinterface interaction method for a handheld device and a television ofclaim 9, wherein the operation mode comprises a cursor mode and a touchmode.
 11. The user interface interaction method for a handheld deviceand a television of claim 10, wherein the touch event in the step ofreceiving a touch event from a user comprises a press event and a moveevent.
 12. The user interface interaction method for a handheld deviceand a television of claim 11, wherein when the touch event is the pressevent, the step of receiving a touch event from a user comprises:determining whether to switch the operation mode according to whetherthe coordinates of the press event are in an operation mode switch area;and determining whether the operation mode is the cursor mode, and ifso, saving the coordinates of the press event without sending the touchevent, and if not, sending the touch event, updating the UI display dataand sending it to the television.
 13. The user interface interactionmethod for a handheld device and a television according to claim 11,wherein when the touch event is the move event, the step of receiving atouch event from a user comprises: determining whether the operationmode is the cursor mode, and if not, sending the touch event, updatingthe UI display data and sending it to the television; and determiningwhether a moving distance of the move event is larger than a threshold,and if not, not processing the move event, and if so, then entering acursor moving state, calculating and moving a display position of thecursor, updating the UI display data and sending it to the television.14. The user interface interaction method for a handheld device and atelevision according to claim 12, wherein in the step of receiving atouch event from a user, when the touch event is the press event, andthe coordinates of the press event are in the operation mode switcharea, the operation mode is then switched to the touch mode.
 15. Theuser interface interaction method for a handheld device and a televisionaccording to claim 12, wherein the operation mode switch area isarranged at the top left of a touch screen of the handheld device. 16.The method of claim 1, wherein the operation mode is switched back andforth from the touch mode to the cursor mode by a tap on a touch screenof the handheld device.
 17. The system of claim 4, wherein the operationmode is switched back and forth from a touch mode to a cursor mode by atap on a touch screen of the handheld device.
 18. The method of claim 9,wherein the operation mode is switched back and forth from a touch modeto a cursor mode by a tap on a touch screen of the handheld device. 19.The method of claim 1, wherein a resolution of the handheld device ismatched with a resolution of the television.
 20. The method of claim 9,wherein a resolution of the handheld device is matched with a resolutionof the television.